Incoming tier wise distribution : This shows the contribution of each of the tiers( startegic ) for the entire population ( qualified and non qualified ) across different vintages.
Note :
Qualified Tier Wise Distribution: This shows the contribution of each of the tiers( startegic ) for the qualified population across different vintages.
Note:
Qualified vs Non-Qualified overtime: This shows the change in proportion of qualified population across vintages.
Note:
If there is any change in qualification rates across the Vintages this can be due to
Tier wise qualification accross different vintages tab )Qualified to born on date difference: This divides the qualified population into three sub categories based on the number of days taken for qualification.
same day of born on datewith in 15 days after born on dateafter 15 days of born on dateNote: If there is no change in the type of population which we are aquiring but if there is increase or decrease in the qualification rate. This can be caused due to people who are not qualified may be getting qualified.
Tier wise qualification across vintages: This shows the proportion of the qualified population with in each tier across different vintages.
Note:
maintain or higher qualification rates in lower risk tiers and lower qualification rate in the higer risk tiers. Any change in any of those qualification proportions needed to flagged.| common_date | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 |
|---|---|---|---|---|---|---|---|---|---|---|
| 2018-08-01 | NA | 0.97 | 0.96 | 0.95 | 0.93 | 0.89 | 0.82 | 0.39 | 0.22 | 0.02 |
| 2018-09-01 | 1 | 0.99 | 0.98 | 0.95 | 0.91 | 0.88 | 0.76 | 0.44 | 0.23 | 0.01 |
| 2018-10-01 | NA | 0.95 | 0.95 | 0.93 | 0.92 | 0.87 | 0.77 | 0.53 | 0.37 | 0.00 |
| 2018-11-01 | NA | 0.96 | 0.95 | 0.93 | 0.91 | 0.90 | 0.81 | 0.53 | 0.43 | 0.00 |
| 2018-12-01 | NA | 0.95 | 0.96 | 0.94 | 0.92 | 0.89 | 0.81 | 0.58 | 0.41 | 0.00 |
| 2019-01-01 | NA | 0.95 | 0.97 | 0.93 | 0.92 | 0.89 | 0.77 | 0.56 | 0.37 | NA |
| 2019-02-01 | NA | 0.96 | 0.96 | 0.94 | 0.91 | 0.89 | 0.77 | 0.61 | 0.49 | 0.00 |
| 2019-03-01 | NA | 0.97 | 0.96 | 0.92 | 0.92 | 0.89 | 0.78 | 0.65 | 0.45 | 0.01 |
| 2019-04-01 | NA | 0.96 | 0.94 | 0.94 | 0.92 | 0.91 | 0.78 | 0.67 | 0.46 | 0.00 |
| 2019-05-01 | NA | 0.96 | 0.95 | 0.94 | 0.92 | 0.91 | 0.81 | 0.62 | 0.46 | NA |
Definition: Population Stability Index ( PSI ) compares the distribution of output variable (predicted probability) of incoming population to an OOT data set that was used during the model development.
Purpose: Especially in the finance domain, model might be influenced by economic changes. PSI gives an idea about how much the distribution of scores have changed from the time of development. If the distibution has significantly changed over time, the model or strategy should be tweeked/changed accordingly.
Rules:
| PSI Value | Inference | Action |
|---|---|---|
| < 0.1 | Insignificant Change | No action required |
| > 0.1 and <0.25 | Some Minor Change | Check other scorecard monitoring metrics |
| > 0.25 | Major shift in population | Need to delve deeper |
For the calculation of PSI, the incoming population has be divided into tiers on two levels:
The purpose of generating PSI on two levels is to cross-validate the threshold values. If there is a change in distribution on both levels, we can conclude that there is a change. Otherwise, the threshold values for the tiers should be looked into as psi is sensitive to the bin-size.
The PSI calculation is further divided into based upon:
Note:
Click here for more information about PSI
| Vintage | psi_val | psi_roll_avg_3 | psi_roll_avg_6 |
|---|---|---|---|
| Aug-18 | 0.1 | -0.01 | -0.01 |
| Sep-18 | 0.08 | -0.01 | -0.01 |
| Oct-18 | 0.06 | 0.08 | -0.01 |
| Nov-18 | 0.09 | 0.08 | -0.01 |
| Dec-18 | 0.08 | 0.08 | -0.01 |
| Jan-19 | 0.07 | 0.08 | 0.08 |
| Feb-19 | 0.05 | 0.07 | 0.07 |
| Mar-19 | 0.07 | 0.06 | 0.07 |
| Apr-19 | 0.08 | 0.07 | 0.07 |
| May-19 | 0.07 | 0.07 | 0.07 |
| Vintage | psi_val | psi_roll_avg_3 | psi_roll_avg_6 |
|---|---|---|---|
| Aug-18 | 0.53 | -0.01 | -0.01 |
| Sep-18 | 0.64 | -0.01 | -0.01 |
| Oct-18 | 0.65 | 0.61 | -0.01 |
| Nov-18 | 0.69 | 0.66 | -0.01 |
| Dec-18 | 0.73 | 0.69 | -0.01 |
| Jan-19 | 1.13 | 0.85 | 0.73 |
| Feb-19 | 0.73 | 0.86 | 0.76 |
| Mar-19 | 0.65 | 0.84 | 0.76 |
| Apr-19 | 0.72 | 0.7 | 0.78 |
| May-19 | 1.11 | 0.83 | 0.85 |
| Vintage | psi_val | psi_roll_avg_3 | psi_roll_avg_6 |
|---|---|---|---|
| Aug-18 | 0.44 | -0.01 | -0.01 |
| Sep-18 | 0.39 | -0.01 | -0.01 |
| Oct-18 | 0.36 | 0.4 | -0.01 |
| Nov-18 | 0.36 | 0.37 | -0.01 |
| Dec-18 | 0.35 | 0.36 | -0.01 |
| Jan-19 | 0.37 | 0.36 | 0.38 |
| Feb-19 | 0.36 | 0.36 | 0.36 |
| Mar-19 | 0.35 | 0.36 | 0.36 |
| Apr-19 | 0.38 | 0.36 | 0.36 |
| May-19 | 0.36 | 0.36 | 0.36 |
| Vintage | psi_val | psi_roll_avg_3 | psi_roll_avg_6 |
|---|---|---|---|
| Aug-18 | 1.38 | -0.01 | -0.01 |
| Sep-18 | 1.42 | -0.01 | -0.01 |
| Oct-18 | 1.36 | 1.39 | -0.01 |
| Nov-18 | 1.41 | 1.4 | -0.01 |
| Dec-18 | 1.42 | 1.4 | -0.01 |
| Jan-19 | 1.49 | 1.44 | 1.41 |
| Feb-19 | 1.33 | 1.41 | 1.4 |
| Mar-19 | 1.23 | 1.35 | 1.37 |
| Apr-19 | 1.41 | 1.32 | 1.38 |
| May-19 | 1.4 | 1.35 | 1.38 |
KS and AUC statistics measures the ability of the model to differentiante between events and non events.
AUC KS summarised : This shows a comparison between the expected KS/AUC and the KS/AUC for the simulated OOT sample ascross different vintages.
Note:
For more details regarding the calculation about the AUC and KS refer:
bad/goods rates accross different tiers for each Vintagebad/goods rates accross different tiers for each VintageNote: - Rolling averages can be used to add significance to the trends in the badrates.
| qualify_vintage | auc_expected | auc_actual |
|---|---|---|
| Aug-18 | 0.64 | 0.66 |
| Sep-18 | 0.66 | 0.67 |
| Oct-18 | 0.66 | 0.67 |
| Nov-18 | 0.68 | 0.65 |
| Dec-18 | 0.65 | 0.59 |
| Jan-19 | 0.67 | 0.64 |
| Feb-19 | 0.67 | 0.77 |
| Mar-19 | 0.66 | 0.58 |
| qualify_vintage | ks_expected | ks_actual |
|---|---|---|
| Aug-18 | 0.21 | 0.27 |
| Sep-18 | 0.23 | 0.25 |
| Oct-18 | 0.23 | 0.29 |
| Nov-18 | 0.28 | 0.21 |
| Dec-18 | 0.23 | 0.19 |
| Jan-19 | 0.26 | 0.21 |
| Feb-19 | 0.28 | 0.43 |
| Mar-19 | 0.25 | 0.18 |
| qualify_vintage | auc_expected | auc_actual |
|---|---|---|
| Oct-18 | 0.65 | 0.67 |
| Nov-18 | 0.67 | 0.65 |
| Dec-18 | 0.66 | 0.59 |
| Jan-19 | 0.67 | 0.64 |
| Feb-19 | 0.66 | 0.77 |
| Mar-19 | 0.67 | 0.58 |
| qualify_vintage | ks_expected | ks_actual |
|---|---|---|
| Oct-18 | 0.22 | 0.29 |
| Nov-18 | 0.25 | 0.21 |
| Dec-18 | 0.25 | 0.19 |
| Jan-19 | 0.27 | 0.21 |
| Feb-19 | 0.25 | 0.43 |
| Mar-19 | 0.26 | 0.18 |
| qualify_vintage | auc_expected | auc_actual |
|---|---|---|
| Jan-19 | 0.66 | 0.64 |
| Feb-19 | 0.67 | 0.77 |
| Mar-19 | 0.67 | 0.58 |
| qualify_vintage | ks_expected | ks_actual |
|---|---|---|
| Jan-19 | 0.25 | 0.21 |
| Feb-19 | 0.26 | 0.43 |
| Mar-19 | 0.26 | 0.18 |
| qualify_vintage | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 |
|---|---|---|---|---|---|---|---|---|---|---|
| 2018-08-01 | 0 | 0.027 | 0.069 | 0.082 | 0.135 | 0.173 | 0.260 | 0.137 | 0.226 | 0.294 |
| 2018-09-01 | 0 | 0.011 | 0.036 | 0.068 | 0.161 | 0.049 | 0.144 | 0.203 | 0.198 | 0.338 |
| 2018-10-01 | 0 | 0.021 | 0.039 | 0.046 | 0.076 | 0.097 | 0.192 | 0.126 | 0.141 | 0.000 |
| 2018-11-01 | 0 | 0.017 | 0.039 | 0.084 | 0.072 | 0.114 | 0.092 | 0.154 | 0.182 | 0.000 |
| 2018-12-01 | 0 | 0.009 | 0.080 | 0.028 | 0.107 | 0.085 | 0.079 | 0.087 | 0.101 | 0.000 |
| 2019-01-01 | 0 | 0.010 | 0.041 | 0.055 | 0.053 | 0.084 | 0.069 | 0.204 | 0.057 | 0.000 |
| 2019-02-01 | 0 | 0.027 | 0.038 | 0.014 | 0.043 | 0.060 | 0.075 | 0.111 | 0.347 | 0.000 |
| qualify_vintage | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 |
|---|---|---|---|---|---|---|---|---|---|---|
| 2018-10-01 | 0 | 0.021 | 0.051 | 0.067 | 0.125 | 0.111 | 0.205 | 0.147 | 0.189 | 0.314 |
| 2018-11-01 | 0 | 0.017 | 0.038 | 0.063 | 0.106 | 0.084 | 0.149 | 0.153 | 0.173 | 0.000 |
| 2018-12-01 | 0 | 0.017 | 0.050 | 0.050 | 0.085 | 0.098 | 0.131 | 0.126 | 0.144 | 0.310 |
| 2019-01-01 | 0 | 0.012 | 0.053 | 0.054 | 0.080 | 0.095 | 0.081 | 0.148 | 0.123 | 0.000 |
| 2019-02-01 | 0 | 0.014 | 0.055 | 0.036 | 0.076 | 0.079 | 0.075 | 0.134 | 0.166 | 0.000 |
| qualify_vintage | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 |
|---|---|---|---|---|---|---|---|---|---|---|
| 2019-01-01 | 0 | 0.017 | 0.052 | 0.063 | 0.108 | 0.105 | 0.161 | 0.147 | 0.159 | 0 |
| 2019-02-01 | 0 | 0.016 | 0.045 | 0.052 | 0.094 | 0.082 | 0.122 | 0.145 | 0.170 | 0 |
| qualify_vintage | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 |
|---|---|---|---|---|---|---|---|---|---|
| Aug-18 | 0.015 | 0.057 | 0.058 | 0.104 | 0.151 | 0.222 | 0.069 | 0.133 | 0.138 |
| Sep-18 | -0.001 | 0.023 | 0.044 | 0.13 | 0.027 | 0.106 | 0.135 | 0.105 | 0.182 |
| Oct-18 | 0.009 | 0.026 | 0.022 | 0.045 | 0.075 | 0.154 | 0.058 | 0.048 | -0.156 |
| Nov-18 | 0.005 | 0.026 | 0.06 | 0.041 | 0.092 | 0.054 | 0.086 | 0.089 | NA |
| Dec-18 | -0.003 | 0.068 | 0.004 | 0.076 | 0.063 | 0.041 | 0.019 | 0.008 | -0.156 |
| Jan-19 | -0.002 | 0.028 | 0.031 | 0.022 | 0.062 | 0.031 | 0.136 | -0.036 | NA |
| Feb-19 | 0.015 | 0.026 | -0.01 | 0.012 | 0.038 | 0.037 | 0.043 | 0.254 | -0.156 |
| qualify_vintage | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 |
|---|---|---|---|---|---|---|---|---|---|
| Oct-18 | 0.009 | 0.038 | 0.043 | 0.094 | 0.089 | 0.167 | 0.079 | 0.096 | 0.158 |
| Nov-18 | 0.005 | 0.026 | 0.039 | 0.075 | 0.062 | 0.111 | 0.085 | 0.08 | NA |
| Dec-18 | 0.005 | 0.038 | 0.026 | 0.054 | 0.076 | 0.093 | 0.058 | 0.051 | 0.154 |
| Jan-19 | 0 | 0.04 | 0.03 | 0.049 | 0.073 | 0.043 | 0.08 | 0.03 | NA |
| Feb-19 | 0.002 | 0.042 | 0.012 | 0.045 | 0.057 | 0.037 | 0.066 | 0.073 | -0.156 |
| qualify_vintage | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 |
|---|---|---|---|---|---|---|---|---|---|
| Jan-19 | 0.005 | 0.039 | 0.039 | 0.077 | 0.083 | 0.123 | 0.079 | 0.066 | NA |
| Feb-19 | 0.004 | 0.032 | 0.028 | 0.063 | 0.06 | 0.084 | 0.077 | 0.077 | NA |
| tier | bad_rate |
|---|---|
| 1 | 0.012 |
| 2 | 0.012 |
| 3 | 0.024 |
| 4 | 0.031 |
| 5 | 0.022 |
| 6 | 0.038 |
| 7 | 0.068 |
| 8 | 0.093 |
| 9 | 0.156 |